监督学习通常依赖于真实标签的手动注释。当有许多潜在的类别时，寻找最佳的班级对于人类注释者可能会过时。另一方面，比较两个候选标签通常要容易得多。我们专注于这种成对的监督，并询问如何有效地用于学习，尤其是在积极学习中。在这种情况下，我们获得了一些有见地的结果。原则上，可以使用$ K-1 $ Active查询来找到最好的$ K $标签。我们表明，有一种自然阶级，这种方法是最佳选择的，并且有更具比较的主动学习方案。我们分析中的一个关键要素是真实分布的“标签邻域图”，如果两个类共享决策边界，则在两个类之间具有优势。我们还表明，在PAC设置中，成对比较在最坏情况下不能提供改善的样品复杂性。我们通过实验补充了理论结果，清楚地证明了邻里图对样品复杂性的影响。

谷歌的运营洪水预测系统是制定的，为机构和公众提供准确的实时洪水警告，重点是河流洪水在大型潮流的河流中。它在2018年开始运作，自从地理位置扩展以来。该预测系统由四个子系统组成：数据验证，阶段预测，淹没建模和警报分配。机器学习用于两个子系统。阶段预测采用长短期内存（LSTM）网络和线性模型进行建模。使用阈值和歧管模型计算洪水淹没，前者计算淹没程度，后者计算淹没程度和深度。本文首次提供的歧管模型提供了一种机器学习替代洪水淹没的液压建模。在评估历史数据时，所有型号都可以实现可操作使用的足够高的度量指标。 LSTM表现出比线性模型更高的技能，而阈值和歧管模型达到了类似的性能度量，以便在淹没程度上进行建模。在2021年的季风季节期间，洪水预警系统在印度和孟加拉国运营，覆盖河流的洪水区，总面积287,000平方公里，拥有350多万人。超过100米的洪水警报被发送给受影响的人口，相关当局以及紧急组织。系统上的当前和未来的工作包括将覆盖范围扩展到额外的洪水易发位置，以及提高建模能力和准确性。

图像分类模型可以取决于图像的多个不同语义属性。对分类器的决定的说明需要对这些属性进行发现和可视化这些属性。在这里，我们通过训练生成模型来具体解释基于分类器决策的多个属性来实现这一点的样式x。此类属性的自然来源是样式语的风格，已知在图像中生成语义有意义的维度。但是，由于标准GaN训练不依赖于分类器，所以它可能不代表对分类器决定很重要的这些属性，并且风格的尺寸可以表示无关属性。为了克服这一点，我们提出了一种培训程序，该培训程序包括分类器模型，以便学习特定于分类器的风格。然后从该空间中选择解释性属性。这些可用于可视化每个图像改变多个属性的效果，从而提供特定于图像的解释。我们将风格x应用于多个域，包括动物，叶子，面和视网膜图像。为此，我们展示了如何以不同方式修改图像以改变其分类器输出。我们的结果表明，该方法发现与语义上保持良好的属性，生成有意义的图像特定的解释，并且是在用户研究中测量的人为解释。

KL-regularized reinforcement learning from expert demonstrations has proved successful in improving the sample efficiency of deep reinforcement learning algorithms, allowing them to be applied to challenging physical real-world tasks. However, we show that KL-regularized reinforcement learning with behavioral reference policies derived from expert demonstrations can suffer from pathological training dynamics that can lead to slow, unstable, and suboptimal online learning. We show empirically that the pathology occurs for commonly chosen behavioral policy classes and demonstrate its impact on sample efficiency and online policy performance. Finally, we show that the pathology can be remedied by non-parametric behavioral reference policies and that this allows KL-regularized reinforcement learning to significantly outperform state-of-the-art approaches on a variety of challenging locomotion and dexterous hand manipulation tasks.

State-of-the-art language models are often accurate on many question-answering benchmarks with well-defined questions. Yet, in real settings questions are often unanswerable without asking the user for clarifying information. We show that current SotA models often do not ask the user for clarification when presented with imprecise questions and instead provide incorrect answers or "hallucinate". To address this, we introduce CLAM, a framework that first uses the model to detect ambiguous questions, and if an ambiguous question is detected, prompts the model to ask the user for clarification. Furthermore, we show how to construct a scalable and cost-effective automatic evaluation protocol using an oracle language model with privileged information to provide clarifying information. We show that our method achieves a 20.15 percentage point accuracy improvement over SotA on a novel ambiguous question-answering answering data set derived from TriviaQA.

Learned classifiers should often possess certain invariance properties meant to encourage fairness, robustness, or out-of-distribution generalization. However, multiple recent works empirically demonstrate that common invariance-inducing regularizers are ineffective in the over-parameterized regime, in which classifiers perfectly fit (i.e. interpolate) the training data. This suggests that the phenomenon of ``benign overfitting," in which models generalize well despite interpolating, might not favorably extend to settings in which robustness or fairness are desirable. In this work we provide a theoretical justification for these observations. We prove that -- even in the simplest of settings -- any interpolating learning rule (with arbitrarily small margin) will not satisfy these invariance properties. We then propose and analyze an algorithm that -- in the same setting -- successfully learns a non-interpolating classifier that is provably invariant. We validate our theoretical observations on simulated data and the Waterbirds dataset.

Selecting subsets of features that differentiate between two conditions is a key task in a broad range of scientific domains. In many applications, the features of interest form clusters with similar effects on the data at hand. To recover such clusters we develop DiSC, a data-driven approach for detecting groups of features that differentiate between conditions. For each condition, we construct a graph whose nodes correspond to the features and whose weights are functions of the similarity between them for that condition. We then apply a spectral approach to compute subsets of nodes whose connectivity differs significantly between the condition-specific feature graphs. On the theoretical front, we analyze our approach with a toy example based on the stochastic block model. We evaluate DiSC on a variety of datasets, including MNIST, hyperspectral imaging, simulated scRNA-seq and task fMRI, and demonstrate that DiSC uncovers features that better differentiate between conditions compared to competing methods.

Graphons are general and powerful models for generating graphs of varying size. In this paper, we propose to directly model graphons using neural networks, obtaining Implicit Graphon Neural Representation (IGNR). Existing work in modeling and reconstructing graphons often approximates a target graphon by a fixed resolution piece-wise constant representation. Our IGNR has the benefit that it can represent graphons up to arbitrary resolutions, and enables natural and efficient generation of arbitrary sized graphs with desired structure once the model is learned. Furthermore, we allow the input graph data to be unaligned and have different sizes by leveraging the Gromov-Wasserstein distance. We first demonstrate the effectiveness of our model by showing its superior performance on a graphon learning task. We then propose an extension of IGNR that can be incorporated into an auto-encoder framework, and demonstrate its good performance under a more general setting of graphon learning. We also show that our model is suitable for graph representation learning and graph generation.

Offline reinforcement-learning (RL) algorithms learn to make decisions using a given, fixed training dataset without the possibility of additional online data collection. This problem setting is captivating because it holds the promise of utilizing previously collected datasets without any costly or risky interaction with the environment. However, this promise also bears the drawback of this setting. The restricted dataset induces subjective uncertainty because the agent can encounter unfamiliar sequences of states and actions that the training data did not cover. Moreover, inherent system stochasticity further increases uncertainty and aggravates the offline RL problem, preventing the agent from learning an optimal policy. To mitigate the destructive uncertainty effects, we need to balance the aspiration to take reward-maximizing actions with the incurred risk due to incorrect ones. In financial economics, modern portfolio theory (MPT) is a method that risk-averse investors can use to construct diversified portfolios that maximize their returns without unacceptable levels of risk. We integrate MPT into the agent's decision-making process to present a simple-yet-highly-effective risk-aware planning algorithm for offline RL. Our algorithm allows us to systematically account for the \emph{estimated quality} of specific actions and their \emph{estimated risk} due to the uncertainty. We show that our approach can be coupled with the Transformer architecture to yield a state-of-the-art planner for offline RL tasks, maximizing the return while significantly reducing the variance.

政策梯度方法被广泛用于学习控制政策。它们可以轻松地分配给多名工人，并在许多领域中达到最新结果。不幸的是，它们表现出很大的差异，随后遭受了高样本的复杂性，因为它们在整个轨迹上汇总了梯度。在另一个极端情况下，计划方法，例如树木搜索，使用考虑未来LookAhead的单步过渡来优化策略。这些方法主要用于基于价值的算法。基于计划的算法需要一个正向模型，并且在每个步骤上都是计算密集型的，但更有效。在这项工作中，我们介绍了SoftTreemax，这是将树搜索整合到策略梯度中的第一种方法。传统上，针对单个状态行动对计算梯度。取而代之的是，我们基于树的策略结构在每个环境步骤中利用树叶的所有梯度。这使我们能够将梯度的差异减少三个数量级，并与标准策略梯度相比，从更好的样本复杂性中受益。在Atari上，与分布式PPO相比，SoftTreemax在运行时的表现高达5倍。